Directing Development Effort with Simulated Students

Our goal is to find a methodology for directing development effort in an intelligent tutoring system (ITS). Given that ITS have several AI reasoning components, as well as content to present, evaluating them is a challenging task. Due to these difficulties, few evaluation studies to measure the impact of individual components have been performed. Our architecture evaluates the efficacy of each component of an ITS and considers the impact of a particular teaching goal when determining whether a particular component needs improving. For our AnimalWatch tutor, we found that for certain go als the tutor itself, rather than its reasoning components, needed improvement. We have found that it is necessary to know what the system’s teaching goals are before deciding which component is the limiting factor on performance.